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Cache Replacement 


[T] Introduction 


In a review by Irani and Karlin (see [^, Chapter 13), they state that although online prob¬ 
lems are pervasive, the cache replacement problem (they refer to it as the “paging prob¬ 
lem” in a two-level store) is the most fundamental and practically important online problem 
in computer science. Megiddo and Modha 12 call caching a “fundamental metaphor in 
modern computing”. 

In a seminal paper, Sleator and Tarjan 14 introduced the concept of competitive 
analysis to analyze the performance of online algorithms by comparing them against the 
performance of an optimal offline algorithm (Opt). They used their model to analyze 
online algorithms such as cache replacement algorithms, proving two important results in 
the process. First, they showed that under the “demand paging” model (no prefetching), 
no online algorithm for cache replacement could achieve a competitiveness ratio less than 
N, where N is the size of the cache. Second, they showed that the Least Recently Used 
(Lru) cache replacement scheme has a competitiveness ratio of N, suggesting that under 
this measure, Lru is optimal. These results are significant. However, we note an important, 
but subtle point that highlights the difference between theory and practice. Sleator and 
Tarjan’s techniques analyze online algorithms in terms of their worst-case behavior (i.e., over 
all possible inputs), which means that other algorithms with poorer competitiveness ratios 
could perform better in practice. Another way to state this is that the results assume an 
oblivious adversary who designs the inputs for the online algorithms in a way that make them 
perform as poorly as possible. The upper bound on performance ratio merely guarantees 
that no surprises are in store, i.e., there is no input designed by an adversary that can make 
the algorithm perform poorly. 


The Lru algorithm was considered the most optimal page replacemement policy for 
a long time, but it had the drawback of not being “scan-resistant”, i.e., items used only 
once could pollute the cache and diminish its performance. Furthermore, Lru is difficult 
to implement efficiently, since moving an accessed item to the front of the queue is an 
expensive operation, first requiring locating the item, and then requiring data moves that 
could lead to unacceptable cache contention if it is to be implemented consistently and 
correctly. The Clock algorithm was invented by Frank Corbato in 1968 as an efficient 
one-bit approximation to LRUwith minimum overhead and continues to be used in MVS, 
Unix, Linux, and Windows operating systems [^. Like Lru, Clock is also not scan- 
resistant becasue it puts too much emphasis on “recency” of access and pays no attention 
to “frequency” of access. So there are sequences in which many other algorithms can have 
significantly less cost than the theoretically optimal Lru. Since then, many other cache 
replacement strategies have been developed and have been showed to be better than Lru 
in practice. These are discussed below in Section 


An important development in this area was the invention of adaptive algorithms. 
While regular “online” algorithms are usually designed to respond to input requests in an 
optimal manner, these self-tuning algorithms are capable of adapting to changes in the 
request sequence caused by changes in the workloads. It adapts to increase or decrease in 
the number of items that have been accessed more than once. Modha and Megiddo 11 


developed a self-tuning algorithm called Adaptive Replacement Strategy (Arc), a hybrid of 
Lfu and Lru. Soon afterwards, another algorithm called Car was also developed. Car is 
a hybrid of Lfu and Clock [^. It had the adaptive flavor of Arc, but was more efficient. 
Experiments show that Arc and Car outperform Lru and Clock for many benchmark 
data sets [^. Versions of Arc have been deployed in commercial systems such as the IBM 
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DS6000/DS8000, Sun Microsystems’s ZFS, and in PostgreSQL. 

Unfortunately, no theoretical analysis of the various adaptive algorithms exists in 
the literature. The main open question that remained unanswered was whether or not there 
exists some “pathological” request sequence that could force Arc or Car to perform poorly. 
We settle this open question by showing that Arc is 4A-competitive and Car is 21fV- 
competitive, thus proving that under Sleator and Tarjan’s theoretical model, their worst- 
case behavior is asymptotically optimal, and therefore not much worse than the theoretically 
optimal Lru. The main challenge in solving these problems is that of carefully designing 
the potential function for the analysis. 

After providing relevant background on cache replacement algorithms in Section we 
discuss the competitive analysis of Arc in Sectionj^and the competitive analyses of Clock 
and Car in Section Concluding remarks can be found in Section 


2 Previous Work on Cache Replacement Strategies 


In practice, since Lru evicts the least recently used entry, it tends to perform well when 
there are many items that are requested more than once in a relatively short period of time, 
and perfomrs poorly on “scans”. Clock implements a circular buffer for the entries, and 
its replacement strategy involves cycling through the pages in that buffer, treating it like a 
clock. Instead of the time of reference as in Lru, Clock maintains a reference bit for each 
item. If an entry is requested, the reference bit is set. Each entry is considered for eviction 
when the clock hand points to it. It is evicted only if its reference bit is not set, else the 
reference bit is reset and the clock moves on to the next entry. 

The DuelingClock algorithm is like Clock but keeps the clock hand at the newest 
page rather than the oldest one, which allows it to be scan-resistant. More recent algorithms 
try to improve over Lru by implementating multiple cache levels and leveraging history. 
In 13 the Lru-A algorithm was introduced. Briefly, the Lru-A algorithm estimates in¬ 


terarrival times from observed requests, and favors retaining pages with shorter interarrival 
times. Experiments have shown Lru- 2 performs better than Lru, and that Lru-A does not 
show increase in performance over Lru- 2 13 , but has a higher implementation overhead. 


It was also argued that Lru-A is optimal under the independence reference model (IRM) 
among all algorithms A that have limited knowledge of the K most recent references to a 


page and no knowledge of the future 13 


In essence, the Lru-A algorithm tries to efficiently approximate Least Frequently Used 
(Lfu) cache replacement algorithm. As K becomes larger, it gets closer and closer to Lfu. It 
has been argued that Lfu cannot adapt well to changing workloads because it may replace 
currently “hot” blocks instead of “cold” blocks that had been “hot” in the past. Lfu is 
implemented as a heap and takes O(logiV) time per request. 

Another cache replacement algorithm is Lirs [^. The Lirs algorithm evicts the page 
with the largest IRR (inter-reference recency). It attempts to keep a small (« 1%) portion of 
the cache for HIR (high inter-reference) pages, and a large (~ 99%) portion of the cache for 
LIR (low inter-reference) pages. The Clock-Pro algorithm approximates Lirs efficienctly 
using Clock [^. The 2 q algorithm is scan-resistant. It keeps a FIFO buffer Ai of 
pages that have been accessed once and a main Lru buffer Am of pages accessed more than 
once. 2 q admits only hot pages to the main buffer. The buffer Ai is divided into a main 
component that keeps the pages in Ai that still reside in cache, and a history component 
that remembers pages that have been evicted after one access. The relatvei sizes of the main 
and history components are tunable parameters. 2 q has time complexity of 0(1). Another 
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algorithm that tries to bridge the gap between recency and frequency is Lrfu 10 . This 


is a hybrid of Lru and Lfu and is adaptive to changes in workload. The time complexity 
ranges from 0(1) for Lru to O(logn) for Lfu. 
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Analyzing ARC 


To facilitate our discussion, we briefly describe the Arc algorithm. As mentioned before, 
it is combines ideas of recency and frequency. Arc’s cache is organized into a “main” part 
(of size N) and a “history” part (of size N). The main part is further divided into two lists, 
Ti and T 2 , both sorted by “recency”. Ti focuses on “recency” because it contains pages 
with short-term utility. Consequently, when an item is accessed for the first time from the 
disk, it is brought into Ti. Items “graduate” to T 2 when they are accessed more than once. 
Thus, T 2 deals with “frequency” and stores items with long-term utility. Additionally, Arc 
maintains a history of N more items, consisting of Bi, i.e., items that have been recently 
deleted from Ti, and B 2 , i.e., items that have been recently deleted from T 2 . History lists are 
also organized in the order of recency of access. The unique feature of Arc is its self-tuning 
capability, which makes it scan-resistant. Based on a self-tuning parameter, p, the size of 
Ti may grow or shrink relative to the size of T 2 . The details of the algorithm are fairly 
complex and non-intuitive. Detailed pseudocode for Arc (Figure 4 from [11| ) is provided 
in the Appendix for convenience. 

It must be noted that we make one minor change in the Arc algorithm as presented by 
Megiddo and Modha. In the original algorithm, when an item is missing from the cache, 
but is found in the history, then the adaptive parameter p increases or decreases depending 
on whether it was found in Bi or ^ 2 . The change is assumed to be by a quantity S, given 
by -|-max{|i 32 |/|i?i|, 1} or — max{|i?i|/|i? 2 |, 1}. For our analysis, we replace the change by 
the quantities -1-1 and —1 respectively. At this point, it is unclear if our analysis extends to 
the original 


3.1 Definitions and Notation 

We start with some notation and definitions. If X is the set of pages in a cache, then let 
MRU{X) and LRU{X) be the most recently and least recently used pages from X. Let 
MRUk{X) and LRUk{X) be the k most recently and k least recently used pages from X. 

Let lists Li (and L 2 ) be the lists obtained by concatenating lists Ti and Bi {T 2 and 
B 2 , resp.). We let ^ 1 , £ 2 , ^i, ^ 2 , &i, denote the sizes of Li, L 2 ,Ti,T 2 , Bi, B 2 , respectively. 
Finally, let t := ti + t 2 and £ := £i + £ 2 - 

In what follows, we will assume that the two algorithms Opt and Arc are run in parallel, 
each maintaining their own caches as per their own replacement policies. At any instant of 
time during the parallel simulation of Opt and Arc, and for any list X, we let MRUk{X) be 
denoted by TOP{X), where k is the largest integer such that all pages of MRUk{X) are also 
in the cache maintained by OPT. We let L[, L 2 ,Tl,T 2 denote the TOPs of Li, L 2 ,Ti,T 2 , 
respectively, with sizes £[,£ 2 ,t[,t 2 , respectively. We let b[ and h '2 denote the sizes of the 
B[ — L'^ n Bi and B 2 = L 2 r\ B 2 , respectively. Note that if b'^ > 0 (63 > 0i resp.), then all 
of Ti (T 2 , resp.) is in Opt. Finally, we let £' := £[ + £' 2 . 

For all our proofs, we assume that algorithm X being analyzed is provided an arbitrary 
request sequence a = cti , 172 , • ■ •, o'm ■ Following the strategy outlined by Albers , we 
partition a into phases P(0), P(l),..., such that X has exactly N faults on P{i), for every 
i > 0. Also, let the subsequence of the request sequence a in phase P{i) be 
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3.2 Analyzing the Competitiveness of ARC 

Let Care and Copt be the costs incurred by the algorithms Arc and Opt. Below, we 
prove the competitiveness of Arc. In any given phase P{i), if the faults are to N distinct 
pages, and if it is different from the last fault prior to the start of phase P{i), then Opt 
is guaranteed to have a fault, allowing for the competitiveness of Arc to be easily proved. 
However, since this is not guaranteed, a more complicated proof is warranted. 

We define the potential function as follows: 

d) = p — [(b'l — t) + 2 ■ (t'l — t) + 3 • {b2 — t) + 4 ■ {t '2 — t)] (1) 

The main result of this section is the following theorem: 

► Theorem 1. Algorithm ARC is AN-competitive. 

We will prove the above theorem by proving the following inequality for any phase P(i), f > 0. 
Note that A A represents the change in quantity X. 

C'arc(CT(f)) + A$ < 4A • C'opt(cr(z)) (2) 

Summing up the above inequality for all requests in all phases would prove the theorem. 
We first state an observation about phase T’(O) as a lemma. 

► Lemma 2. At the end of phase P{0), Arc’s eache is “full”, i.e., t = N. The cache 
remains full from that point onward. 

Proof. Note that in phase P{0), the request sequence (t( 0) requests N distinct pages. At 
the end of phase P(0), the cache maintained by Arc is full, i.e., t = N. Since evictions only 
happen when Arc has a miss, its cache will remain full from this point onward. ◄ 

In phase P(0), if Arc and Opt start with the same cache contents, then the first time 
Arc faults in this phase, so will Opt, thereby guaranteeing that Opt faults at least once 
during this phase. Since Arc faults exactly N times during this phase, inequality holds 
for phase P(0). 

To analyze phase P{i),i > 0, we will prove that for every individual request, aj G o’(i): 

Carc{.aj) + X<^< AN-Coptic,) (3) 

Summing up the above inequality for all requests, G a{i) will prove inequality Q for 
phase P{i). 

We assume that request cjj is processed in two distinct steps: first when Opt services 
the page request and, next when Arc services the request. We will show that inequality ^ 
is satisfied for each of the two steps. 

Step 1: Opt serves request Uj. 

Since only Opt acts in this step, Carc = 0, and t does not change. There are two 
possible cases: either Opt faults on aj or it does not. 

If Opt does not fault on this request, then Cqpt = 0. Since the contents of the cache 
maintained by Opt does not change, and neither do the quantities b[,b 2 ,t[,t 2 , we have 
Ad) = 0, and Carc{f^ii)) + Ad) < AN * -Copticr^i)) < 0. 

If Opt faults on request cr(i), then Copt = 1- The contents of the cache maintained by 
Opt does change, which will affect the potential function. Since 0 < 5^ + 1[ + 62 + <2 A A, 
the quantity 61 + 2 ^^+ 362 + 4^2 cannot change by more than AN. Thus, Care(o'j) +Ad) < AN, 
proving inequality 
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Step 2: Arc serves request Uj. 

There are four possible cases. Case 1 deals with the case when Arc finds the page in 
its cache. The other three cases assume that Arc faults on this request beause the item is 
not in Ti U T 2 . Cases 2 and 3 assume that the missing page is found recorded in the history 
in lists Bi and B 2 , respectively. Case 4 assumes that the missing page is not recorded in 
history. 

Case I: Arc has a page hit. 

Clearly, Care = 0. We consider several subcases. In each case, the requested page will be 
moved to MRU{T 2 ) while shifting other pages in T 2 down. 

Case 1.1 If the requested page is in T[, the move of this page from T[ to implies At'i = 
-1; M '2 = +1 and A4> = -(2 • + 4 • = -2. 

Case 1.2 If the requested page is in T 2 , the move of this page to MRU{T 2 ) does not change 
the set of items in T^. Thus, At'i = At^ = 0 and Ad* = 0. 

Case 1.3 If the requested page is in Ti — T[, it is similar to the case when the requested 
page is in except that it could cause t[ and b[ to increase by a large amount. This 
could occur if the requested item was immediately following T[ and its move caused items 
below it to become part of T[. Thus we have, At 2 = +1 and At'^, Ab[ > 0. Finally, 

A4> < -{Ab[ + 2At[ + 4At2) 

< -4 

Case 1.4 If the requested page is in T 2 — T 2 , it is similar to the case when the requested 
page is in T 2 , except that it could cause t '2 and b '2 to increase by a large amount. Thus 
we have, A &2 > 0 and At '2 > 1. Finally, 

A4> < -(3A&2 + 4At2) 

< -4 

Next we will analyze the three cases when the requested page is not in Arc’s cache. 
Since Care = 1, file change in potential must be at most -1 in order for inequality ^ to be 
satisfied. We make the following useful observations in the form of lemmas. 

► Lemma 3. If Arc has a miss and if the page is not in Arc’s history, we have C = 
t\ +t '2 + b[ + 62 < A. Consequently, we also have < N and £3 < N. 

Proof. Since Opt has just finished serving the request, the page is present in the cache 
maintained by Opt just before Arc starts to serve the request. If Arc has a miss, there is 
at least one page in the cache maintained by Opt that is not present in the cache maintained 
by Arc, implying that I' < N. By definition, £' = £[ + £2 = t[ + 1 2 + b[ + 62 . Thus, the 
lemma holds. ◄ 

► Lemma 4. A call to procedure REPLACE either causes an element to be moved from Ti 
to Bi or from T 2 to B 2 . In either case, the change in potential due to REPLACE, denoted 
by A<i)/j, has an upper bound of 1. 

Proof. Procedure REPLACE is only called when Arc has a page miss. Clearly, it causes 
an element to be moved from Ti to Bi or from T 2 to R 2 - If that element is in T[ or T 2 , then 
either Ti = T[ or T 2 = the moved element becomes part of B[ or Thus, the 

potential change is +1. If that element is in Ti — T[ {T 2 — T 2 , resp.), then B[ resp.) 
was empty before the move and remains empty after the move. Hence, there is no change 
in potential. ◄ 

► Lemma 5. After phase P(0), every eviction is the result of an Arc miss and involves 
either LRU{Li) or LRU{L 2 ). However, this page will be neither from L'l nor 


Shao et al. 


7 


Proof. By Lemmawe know that at the end of phase P( 0 ), the cache maintained by Arc 
is full, i.e., t = ti + t2 = N. Every miss after that involves moving an item from Ti to Bi 
or from T2 to B2, followed by a possible eviction from either Li or L2- The proof is by 
contradiction. Assume that a page is evicted from either Li n L\ or L2 H 

First, assume that the evicted page is from Li n L\. The only time when Arc will 
remove a page from Li is when ii = i[ = N (Case IV A from the Arc algorithm). By 
Lemma 1 ^ this is impossible since when Arc has a miss, i'l < N. 

Next, assume that the evicted page is from L2nL2, implying that £2 = £'2- The only time 
when Arc will remove a page from L2 is when li < N (case IV B from Arc algorithm), 
implying that that Z2 = ^2 — which is impossible since Opt can have at most N 

items. ◄ 

► Lemma 6. After phase P( 0 ), if Ti = T[ then a page in T2 will not be moved to B2. 
Similarly, if T2 = T2 then a page from T[ will not be moved to Bi. 

Proof. In an attempt to prove by contradiction, let us assume that Ti = T[ and T2 = T2 
are simultaneously true and Arc has a miss. By Lemma the cache maintained by Arc 
is full after phase T’(O). Thus, we have t = ti + £2 = N = t'^ + t'2, which is impossible by 
Lemma IH 

Thus, on an Arc miss, if Ti = T{, then T2 yf T^. Thus if LRU{T2) is moved to B2, this 
item cannot be from T^. By a symmetric argument, if T2 = T2, then Ti ^ T{, and LRU{Ti) 
is not in T(. ◄ 

Case II: Arc has a miss and the missing page is in Bi 

Note that in this case the value of p will change by + 1 , unless its value equals N, in which 
case it has no change. Thus Ap < 1 . 

If the missing item is in B[, then Ab[ = —I and At2 = + 1 . Adding the change due to 
REPLACE, we get 

A$ < 1 - (Afc^ + 4 • At2) + 

< -1 

If the missing item is in Bi — B[, then, as argued before, could increase by a large 
amount if this item immediately follows items in B[ and is also followed by items in Opt. 
Thus, we have At2 = 1 and Ab[ > 0 . Thus, we have 

A 4 > < 1 - (A61 + 4 • At2) + 

< -2 

Case III: Arc has a miss and the missing page is in i?2- 

Note that in this case the value of p will change by - 1 , if its value was positive, otherwise it 
has no change. Thus Ap < 0 . 

If the requested item is in B2, then At^ = 1 , and A&2 = — 1 - Thus, we have 

A 4 > = Ap - (3 • A62 + 4 • At 2 ) + A^h 
< 0 

But this is not good enough since we need the potential change to be at most - 1 . When 
Ap = — 1 , then we get the required inequality Ad* < — 1 . Clealry, the difficulty is when 
Ap = 0 . Since the missing item is from 5 ^, it implies that non-empty and T2 = T2. By 
Lemma|^above, since the cache is full (i.e., ti+t2 = N), there must be at least one item in 
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Ti — T{, which means that means that ti > 0 . As per the algorithm, since Ti is non-empty 
and p = 0 , we are guaranteed to replace LRU{Ti), and not an element from Therefore, 
REPLACE will leave and h\ unchanged, impying that = 0 . Thus, we have 

A$ = Ap - (3 • A62 + 4 • A 4 ) + 

< -1 

If the requested item is from B2 — B2, then At^ > 1 , and A&2 > 0 . Thus, we have 

A$ < Ap - (3 • Ab 2 + 4 • At'2) + A$i^ 

< -3 

Case IV: Arc has a miss and the missing page is not in Bi U B2 

We consider two cases. First, when £i = N, Arc will evict the LRU{Li). Since by Lemma 
1 ^ £!-y < A^, we know that for this case, h\ remains 0 and At[ = -|- 1 . Thus, we have 

A4> < -{2-At[) + A<^ii 

< -1 

On the other hand, if £i < N, then Arc will evict the LRU{L2). Again, if the cache 
is full (i.e., ti + £2 = N and £1+ £2 = 2 A), then we know that £2 > N, which means that 
L2 yf L2 and LRU{L2) is not in Thus, deletion of LRU{L2) = LRU{B2) will not affect 
h'2 or any of the other quantities in the potential function. Then comes the REPLACE step, 
for which a bound has been proved earlier. Finally, a new item is brought in and placed in 
MRUiTi). Thus At'i < 1 . Putting it all together, we have 

A$ < -{2- At\) + A^r 
< -1 

Wrapping up the proof of Theorem 

Tying it all up, we have shown that inequality (|^ holds for every request in the phase 
P{i),i > 0 , proving that inequality ([^ holds for every phase. We showed at the start that 
inequality ([^ also holds for phase P( 0 ). Putting it all together, we have that 

Carder) + A<^> < m ■ Copti^y). 

If we assume that the caches started empty, then the initial potential is 0 , while the final 
potential can be at most AN. Thus, we have 


Carder) < ■ Copder) + AN, 

thus proving Theorem ◄ 

[~^ Analyzing CAR 

As mentioned earlier. Clock with Adaptive Replacement (Car) algorithm is a modification 
of the Clock algorithm and is inspired by the Arc algorithm. We start with a quick 
description of the Clock algorithm. 

Clock was designed as a low-cost variant of Lru, requiring only one bit per item and one 
pointer to implement. Clock’s cache is organized as a single “circular” list. The algorithm 
maintains a pointer to the “head” of the list. The item immediately counterclockwise to it 
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is the “tail” of the list. Each item is associated with a “mark” bit. Some of the pages in the 
cache are marked, and the rest are unmarked. When a page hit occurs that page is marked. 
The contents of the cache remain unchanged. When a page miss occurs, in order to make 
room for the requested page, the head page is evicted if the page is unmarked. If the head 
page is marked, the page is unmarked and the head is moved forward clockwise. After a 
page has been evicted, the requested page is unmarked and placed at the tail of the list. 

Inspired by Arc, Car’s cache is organized into two main lists, Ti and T2, and two 
history lists, Bi and i?2- Inspired by Clock, both Ti and T2 are organized as “circular” 
lists, with each item associated with a mark bit. The history lists, Bi and B2 are maintained 
as simple FIFO lists. We let ti, t2, ^1, ^2 denote the sizes of Ti, T2, i?i, i?2, respectively. Also, 
let t := ti + O- Let lists Li (and L2, reps.) be the list obtained by concatenating list Bi to 
the tail of Ti (concatenating B2 to the tail of T2, resp.), with sizes £i (and £2, resp.). We let 
Ti and T2 denote the unmarked pages in Ti and T2, respectively; similarly, we let Tl and 
T2 denote the marked pages in Ti and T2, respectively. 

The following 7 invariants are maintained by Car for the lists: (II) Q <ti + t2 < N] ( 12 ) 
0 < tl + 61 < A; ( 13 ) 0 < O + 62 < 2 N- ( 14 ) 0 < ti + O + + &2 < 2 N- ( 15 ) If ti + O < A, 

then 61 + 62 = 0 ; ( 16 ) If ti + ^2 + fei + t)2 > A, then ti + ^2 = A; ( 17 ) Once the cache is full, 
it remains full from that point onwards. 

Car maintains an adaptive parameter p, which it uses as a target for ti, the size of 
list Tl. Consequently, N — p is the target for ^2- Using this guiding principle, it decides 
whether to evict an item from Ti or T2 in the event that a miss requires one of the pages 
to be replaced. The replacement policy can be summairzed into two main points: (a) If the 
number of items in Ti (barring the marked items at the head of the list) exceeds the target 
size, p, then remove a page from Ti, else remove a page from T2 or remove one of the marked 
pages at the head of the list; and (b) If £i = h + bi = A, then remove a history page from 
Bi, else remove a history page from B2. Since the details of the algorithm are complex, the 
actual pseudocode is provided (Figure 2 from i) in the Appendix. 

4.1 Analyzing the Competitiveness of CLOCK 

Before proving the competitiveness of Car, since the algorithm relies on the simpler Clock 
algorithm, we first prove the competitiveness of Clock. We are not aware of the existence 
of a formal proof of competitiveness of Clock. Our result is formulated as the following 
theoreim: 

► Theorem 7. Algorithm Clock is 2 N-competitive. 

Proof. Let g be a page requested by the request sequence. We define R[q] as follows. If 
q is in the cache maintained by Clock, but is not part of the cache maintained byOPT, 
then R[q] is computed as follows. If q is unmarked, i?[g] equals the position of q counting 
counterclockwise from the tail. (If item q is at the tail, then R[q] = 1 ). If g is marked, then 
it equals A plus the position from the tail. Otherwise, R[q] = 0 . 

Following the proof for Arc, let Cdock and Copt be the costs incurred by the algorithms 
Clock and Opt, and let a = cti, 0 - 2 , ..., CTm be an arbitrary request sequence. As before, 
we partition a into phases P( 0 ), T’(l),..., such that Clock has exactly A faults on P{i), 
for every i > 0 . Let the subsequence of the request sequence cr in phase P{i) be (j{i). 

Let D be the set of pages that are in the cache maintained by Clock, but not in the 
cache maintained by Opt. We define the potential function as follows: 

d> = ^ R[q] 

qGD 


( 4 ) 
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If both algorithms, Clock and Opt, start with the same cache contents, then phase 
P( 0 ) is easily handled since Opt will fault at least once and Clock faults exactly N times, 
making Clock A^-competitive in this phase. 

In order to analyze phase P(i),i > 0 , we will prove that for every individual request, 
aj e a{i): 


Cclock{<^j) + ^ 2iV * Copt{<^j) 


(5) 


Summing up the above inequality for all requests, aj G a{i) will prove inequality (|^. 

As before, we assume that request aj is processed in two distinct steps: first when Opt 
services the page request and, next when Clock services the request. We will show that 
inequality (§ is satisfied for each of the two steps. 

When only Opt acts in this step, Cdock = 0 , and t does not change. If Opt does not 
fault on this request, then Cqpt = 0 . Since the contents of the cache maintained by Opt 
does not change, no pages get marked or unmarked, and the clock hand does not move. 
Thus, A<i) = 0 . 

If Opt faults on request aj, then Cqpt = 1- The contents of the cache maintained by 
Opt does change, which could affect the potential function. If a marked page in Opt gets 
evicted, since its i?-value cannot exceed 2 N, the potential will change by at most 2 N. Thus, 
Carc{ckj) + A<i) < 2 N, proving inequality 

Next we consider the steps when Clock services the request. If Clock has a hit, then 
there will be no change in potential since pages do not move around. If the requested page 
is marked by Clock, it does not change the potential either since the page is already in 
the cache maintained by Opt and is not considered for the potential function calculations 
Finally, we consider the case when Clock has a miss. Since there are N pages in the cache 
maintained by Clock, at least one of those pages is guaranteed to be not part of the cache 
maintained by Opt. The newly requested page will be moved to the tail position. All other 
pages will move down by at least one position (in which case their i?-value decreases by 1) 
or will lose their marked status (in which case their i?-value decreases by a large amount, 
going down from 2 N to 1). Either way, the decrease in potential will pay for Clock’s miss. 
This completes the proof of the theorem and the competitiveness analysis of the Clock 
algorithm. ◄ 


4.2 Analyzing the Competitiveness of CAR 

Next, we analyze the competitiveness of Car. The main result of this section is the following: 
► Theorem 8. Algorithm Car is 21 N-competitive. 

Proof. We introduce the following notation. Let q be any page in the request sequence. If q 
is in the cache maintained by Car, then it is in one of the lists Ti,T 2 , Bi, B 2 , and we define 
P[q] to be the position of q in that list. For Ti and T2, position is counted from the tail of 
the list to the head of the list in the counterclockwise direction. For Bi and B2, position 
is counted from the MRU to the LRU. Recall that T}* and T2 are the unmarked pages in 
Ti and T2, while and T2 are the marked pages in Ti and T2. Let D be the set of pages 
in Ti U T 2 U i?i U i? 2 , but not in the cache maintained by Opt. Let U be the set of pages 
contained in the intersection of Ti UT 2 and the set of pages in the cache maintained by Opt. 
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As before, we associate each page with a value R[q], which is defined as follows: 


PK 

if 9 G i?i U 

2 ■ P[q] + bi, 

if g G T° 

2-P[q]+b2, 

if q G TO 

3N + 2- P[q] + bi, 

if g G Tl 

3N + 2- P[q] + b 2 , 

if g G Tj 

0 , 

otherwise 


Finally, we define the potential function as follows: 

$ = p+2-(&i+ti)-3-|C7|+3- E Kbl (6) 

qGD 

We prove Theoremby proving the following inequality for any phase P{i),i > 0. 


Carc{<j{l)) + < 21N ■ (7) 

Summing up the above inequality for all requests in all phases would prove the theorem. 
The arguments for phase P{0) are identical to that for ARCand are not reproduced here. In 
order to analyze phase T’(i), i > 0 , we will prove that for every individual request, aj G cr{i): 

CcaricTj) + < 21 iV • Copt{<Jj) ( 8 ) 

Summing up the above inequality for all requests, aj G cr(i) will prove inequality Q for 
phase P(i). As before, we assume that request aj is processed in two distinct steps: first 
when Opt services the page request and, next when Car services the request. We will show 
that inequality ^ is satisfied for each of the two steps. 

Step 1: Opt serves request Uj. 

Since only Opt acts in this step, Ccar = 0, and t does not change. There are two 
possible cases: either Opt faults on aj or it does not. 

If Opt does not fault on this request, then it is easy to see that Copt = 0 and Ad* = 0, 
thus satisfying inequality 

If Opt faults on request a{i), then Cqpt = 1- The contents of the cache maintained 
by Opt changes, with at most one page q whose R[q] value changes. The maximal positive 
change will occur when the marked head page in T 2 is removed from Opt. Additionally, if 
I 22 I + IB 2 I = 2A, then it could result in in a potential change of 18iV. Since one element 
will be evicted from the cache maintained by Opt, |f7| may change by at most 1. Thus, 
the potential function d) may change by at most 18N + 3 < 21N. Ccar{<^j) + A<i) < 21A, 
proving inequality §. 

Step 2: Car serves request aj. 

As with the proof for Arc, there are four cases. While the structure is the same, the 
details are different. Case 1 deals with the case when Car finds the page in its cache. The 
other three cases assume that Car faults on this request beause the item is not in Ti U T 2 . 
Cases 2 and 3 assume that the missing page is found recorded in the history in lists Bi and 
B 2 , respectively. Case 4 assumes that the missing page is not recorded in history. 

Case I: Car has a page hit. 

Clearly, the page was found in Ti U r 2 ) and Ccar = 0. Neither the cache nor the history 
lists maintained by Car will change. Thus the contribution to |f7| does not change. Since 
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the page is not found in Car’s history, Ap = 0. Since Opt has aleady served the page, the 
page is in Opt’s cache. Therefore, even if the page gets marked during this hit, its i?-value 
does not contribute to the potential. Thus, Ad* = 0. 

Next we will analyze the three cases when the requested page is not in Car’s cache. 
Since Ccar = the change in potential must be at most -1 in order for inequality ([^ to be 
satisfied. The following lemma is useful in understanding the potential change to the term 

► Lemma 9. When Car has a page miss, the change to the term most 0. 

Proof. We examine the potential change based on the original location of the page(s) whose 
i?-value changed. 

Case q 6 Bi This implies that page q or some page in Bi that was more recently used 
(higher i?-value) was either discarded or requested (and consequently moved out). The 
contribution from the i?-values of the pages in Si can therefore only go down or remain 
the same. If the page moved to T 2 , then it was because it was accessed, but then it is 
already in Opt and therefore does not contribute. 

Case q 6 T° A page in Tf will decrease in i?-value when bi decreases, or as a result of a 
call to REPLACE. This happens when some page is moved from Ti to Bi or form Ti 
to T 2 (causing all pages to modify their position numbers and their i?-values). However, 
these changes only decrease their i?-values. 

Case q 6 The argument is exactly the same as for the repvious case. In the event that 
the page itself is moved to T 2 it will lose its marked status which will cause a decrease 
in potential. 

•4 

The other three cases are omitted since they are similar to the above three. As with 
Arc’s analysis, we know that if Car has a miss there is at least one page in either Ti or T 2 
that is not part of Opt. We proceed with analyzing the changes in the other terms in the 
potential function. 

Case II: Car has a miss and the missing page is in Bi 

Note that in this case the value of p will change by at most +1, unless its value equals N, 
in which case it has no change. Thus Ap < 1. Ai?[g] will be a non-positive number. The 
reason why it is not guaranteed to be negative is because all the pages ordered above may 
be part of Opt and then REPLACE is called on Ti. In this case R[q] will not change. 
A|Ti| -I- \Bi\ is guaranteed to decrease by at least one. Car and Opt will still share the 
same pages. 

Ad> < 3-A ^ R[q] + 2-A\Bl + Tl\ +Ap-2,-A{Car £Opt) 

q^Car (history) / Opt 

< -1 

Case III: Arc has a miss and the missing page is in i? 2 - 

Note that in this case the value of p will change by -1, if its value was positive, otherwise it 
has no change. Thus Ap < 0. 

If we assume that REPLACE is called on T 2 and R[q] doesn’t then we could have 
Ad> < 3-A^R[g]-k2-A(6i-kti)-l-Ap-3-A|C/| 

qGD 


< 0 
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But this is not good enough since we need the potential change to be at most -1. When 
Ap = —1, then we get the required inequality Ad) < —1. Cleary, the difficulty is when 
Ap = 0. 

Case 111.1 If REPLACE is called on Ti and a page in Ti is not part of Opt then the contents 
of R[q] will change by at least one. Thus Ad* < —1. 

Case 111.2 If REPLACE is called on Ti and all pages in Ti are part of Opt then the contents 
of R[q] won’t change from REPLACE. Since (lemma) R[q] will change from B 2 moving 
to T2. 

Case III.3 If REPLACE is called T 2 this means that R[q] will definitely decrease... 

Case IV: Car has a miss and the missing page is not in Bi U B 2 

We consider three cases. First, when £1 = A, Car will evict the LRU{Bi). Since there are 
N pages in Li at least one of the pages must not be part of Opt. By lemma() we know that 
the potential will change by a negative amount. 

A d>< 3-A^i?[(7] + 2-A(5i+ti)-3-A|[/| 

qGD 

< -1 

If £1 < A and £ = N, then Car will evict the LRU{B 2 ). Since there are N pages in L 2 
at least one of the pages must not be part of Opt. By lemma() we know that the potential 
will change by a negative amount. 

A d>< 3-A^i?[(7] + 2-A(5i+£i)-3-A|££| 

qGD 

< -1 

The last case that will be examined will have no discards from history. Since no pages 
are discarded there is no guarantee that R[q] will change. This also means that Car and 
Opt will have one more page in common. Thus, the equation is 

Ad> < 3-A^R[g]+2-A(6i + £i)-3-A|C/| 

q&D 

< -1 


•4 
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Conclusions 


Adaptive algorithms are tremendously important in situations where inputs are infinite on¬ 
line sequences and no single optimal algorithm exists for all inputs. Thus, different portions 
of the input sequence require different algorithms to provide optimal responses. Conse- 
qunetly, it is incumbent upon the algorithm to sense changes in the nature of the input 
sequence and adapt to these changes. Unfortunately, these algorithms are harder to ana¬ 
lyze. We present an analysis of an importnat adaptive algorithm called Arc and show that 
it is competitive. 
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Pseudocode: ARC(a:) 

INPUT: The requested page x 

INITIALIZATION: Set p = 0 and set lists Ti, Bi, T 2 , and B 2 to empty 

1: if (x is in Ti U T 2 ) then > cache hit 

2: Move X to the top of T 2 . 

3: else if (x is in Bi) then 

4: Adaptation: Update p = min{p +max{l, |i? 2 |/|Bi|},-N} 

5: Replace() 

6: Move X to the top of T 2 and place it in cache. 

7: else if (x is in B 2 ) then 

8: Adaptation: Update: p = max{p — max{l, |i3i|/|i32|},0} 

9: ReplaceO 

10: Move X to the top of T 2 and place it in cache. 

11: else t> cache and directory miss 

12: if (iTil + |i?i| = N) then 

13: if (|Ti| < N) then 

14: Discard LRU page in Bi. 

15: ReplaceO 

16: else 

17: Discard LRU page in Ti and remove it from cache. 

18: end if 

19: else if ((iTil + |Ri| < N) and (|Ti| + IT 2 I + \Bi\ + IR2I > N)) then 

20: if (iTil + \T2\ + \Bi\ + IR2I = 2N) then 

21: Discard LRU page in B 2 . 

22: end if 

23: ReplaceO 

24: end if 

25: end if 


ReplaceO 

26: if ((|Ti| > 1) and {{x is in B 2 and |Ti| = p) or (|Ti| >p))) then 

27: Delete LRU page in Ti (also remove it from cache); move it to MRU position in Bi. 

28: else 

29: Delete LRU page in T2 (also remove it from cache); move it to MRU position in i?2- 

30: end if 
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Pseudocode: CAR(a;) 

INPUT: The requested page x 

INITIALIZATION: Set p = 0 and set lists Ti, Bi, T 2 , and B 2 to empty 


1 

2 

3 

4 

5 

6 

7 

8 
9 

10 

11 

12 

13 

14 

15 

16 

17 

18 

19 

20 
21 


if (x is in Ti U r2) then > cache hit 

Set the page reference bit for x to one. 

else > cache miss 

if (iTil + IT2I = N) then > cache full, replace a page from cache 

Replace() > cache directory replacement 

if ((x is not in Bi U B 2 ) and (|ri| + |i?i| = N)) then 
Discard LRU page in Bi. 

else if (dPil + IT 2 I + \Bi\ + IR2I = 2N) and (x is not in Bi U B 2 )) then 
Discard LRU page in i?2- 
end if 
end if 

if (x is not in Bi U B 2 ) then > cache directory miss 

Insert x at the tail of Ti. Set the page reference bit of x to 0. 
else if (x is in Bi) then > cache directory hit 

Adaptation: Update p = min{p + max{l, |i?2|/|.Bi|}, N} 

Move X at the tail of T 2 . Set the page reference bit of x to 0. 
else i> cache directory hit 

Adaptation: Update: p = max{p — max{l, |Ri|/|R2|},0} 

Move X at the tail of T 2 . Set the page reference bit of x to 0. 
end if 
end if 


Replace() 

22: found = 0 
23: repeat 

24: if (iTil > max(l,p)) then 

25: if (page reference bit of head page in Ti is 0) then 

26: found = 1 

27: Demote head page in Ti and make it MRU page in Bi 

28: else 

29: Set the page reference bit of head page in Ti to 0, and make it tail page in T 2 . 

30: end if 

31: else 

32: if (page reference bit of head page in T 2 is 0) then 

33: found = 1 

34: Demote head page in T 2 and make it MRU page in i? 2 - 

35: else 

36: Set page reference bit of head page in T 2 to 0, and make it tail page in T2. 

37: end if 

38: end if 

39: until (found) 





